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Abstract: In this paper we show some new look at large deviation theorems 
from the viewpoint of the information-spectrum (IS) methods, which has been first 
exploited in information theory, and also demonstrate a new basic formula for the large 
deviation rate function in general, which is expressed as a pair of the lower and upper 
IS rate functions. In particular, we are interested in establishing the general large 
deviation rate functions that can be derivable as the Fcnchel-Legendre transform of 
the cumulant generating function. The final goal is to show a necessary and sufficient 
condition for the rate function to be of Cramer-Gartner-Ellis type. 







1 Introduction 



The present paper is intended to show a basic new look at problems in large deviation 
theory. As is well known, in many fields such as information theory, probability 
theory, computer sciences, communication sciences, and cryptography, etc., we face 
the problem to pertinently evaluate behaviors of the tail probability of asymptotic 
distributions, and it is now not only of theoretical importance but also of practical 
interest. 

In such a situation, therefore, it would also be useful to elucidate and look back 
at basic common properties as well as some common structures underlying the large 
deviation problem. 

For this purpose, we first introduce, as a basic tool for analyzing the tail prob- 
abilities in large deviation, the concept of a general source Z = {Zn}^^i that is 
general in the sense that our scope is not only in stationary/ergodic sources but also 
in non-ergodic and/or nonstationary sources. In order to enable us to deal with this 
kind of general sources, we define a pair of two kinds of large deviation rate func- 
tions, say, a pair of the lower and upper information-spectrum (=IS) rate functions, 
basically without any assumptions on its probabilistic memory structures. This is 
because sources in consideration may be non-ergodic and/or nonstationary. It should 
be noted here that, by convention, large deviation rate functions have been supposed 
to be assigned with one each source. Along this line of thought, we try to reveal a 
general skeleton latent in those basic problems. Such an IS approach to various kinds 
of information-theoretical problems has been devised by Han and Verdii [1], Verdii 
and Han [2], and Han [3], etc. 

Several crucial studies on the fundamental large deviation problem are found in 
Varadhan [7] (the integral lemma), Bryc [8] (the inverse Varadhan lemma), etc. In 
particular, Bryc [8] has shown that a kind of "nonlinear'^ but one-to-one transfor- 
mations (instead of ^Hineaf^ transformations, say, Fenchel-Legendre transformations) 
is quite useful in order to establish the good rate function in general that may not 
necessarily be convex, though the relevant computation of these nonlinear transforms 
seems to be formidable. 

On the other hand, in the present paper, wc start with the introduction of the 
notion of lower /upper IS rate functions to demonstrate that the pertinent large devi- 
ation rate functions (the lower one and the upper one) can be reasonably formulated 
with the use of these IS rate functions. 

In Section 2 we prepare the necessary notion to establish the pair of basic funda- 
mental lower/upper large deviation rate functions, where by a "rate function" we do 
not necessarily mean that it is lower semicontinuous and all the level sets are closed 
(cf. Dembo and Zeitouni [4]). 

In Section 3 it is shown, as illustrative applications of the theorems stated in 
Section 2. that, for mixed somvcqs, the lower and upper IS rate fimctions coincide with 
each other, but they are not convex (cf. Bryc [8], Chen [5]); and also that the lower 
and upper IS rate functions do not coincide with each other for nonstationary sources. 

In Section 4, a pleasant generalization from the real space R = (— cxd, +oo) to the 
Hausdorff topological space is pointed out, although this generalization is not used in 
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the subsequent sections. 

In Section 5, as a second crucial step, we proceed to elucidate a structural corre- 
spondence between the lower/upper IS rate functions and the inferior/superior hmits 
of the normalized cumulant generating functions to demonstrate the formula for com- 
puting the cumulant generating function using the lower /upper IS rate functions, the 
inverse fimction of which, i.e., the rate fimction of Cramcr-Gartner-EUis type is given 
in Theorem 5.4. Here, a conclusion (Theorem 5.5) is stated that these are connected 
equivalently with each other via the Fenchel-Legendre transformation, under a mild 
assumption, if only and if the former IS rate functions are closed and convex, which 
is called the reduction theorem. 

Finally, in Section 6 we give the proofs of Theorem 5.1 and Lemma 5.1. 



2 A General formula for Large Deviations 

Let (^1,^2, • • •) be any sequence of random variables taking values in R= (— oo, -|-oo), 
and call Z = {Zn}^^i a general source. Here, any probabilistic dependency structures 
are basically not assumed about Z = {Zn}^^i, where Zn is supposed typically to be 
the arithmetic mean ^ with another underlying "sum" source S = {5„}^i. We are 
interested in large deviation behaviors of those Z = {Z„}J^^. 

Let TTi > 7r2 > • • • ^ be an arbitrarily proscribed sequence. With this TTj 
{i = 1, 2, • ■ •) define the shrinking open intervals as follows: 

$,(i?) = (i?-7ri,i? + 7r,) (i = l,2,---), (2.1) 

and also define as follows: 



Definition 2.1 



Hi{R) = liminf-log— — ^— — , (2.2) 

— ' n^oo n Pr{Z„ G $i(i?)} ^ ' 

Hi{R) = limsup-log 3^ .pu ' (2-^) 

„^oo n Pr{Z„ e ^i{R)} 



H{R) = lim Hi{R), (2.4) 
H{R) = lim Hi{R). (2.5) 

□ 

Clearly, H_i{R) and Hi{R) arc increasing functions in i. It is also obvious that 
Hi{R) < Hi{R) {i = 1,2,---) and H{R) < H{R). We call these H{R),H{R) the 
lower /upper IS rate functions of Z = {Z„}^^. It should also be remarked that 
H_{R), H{R) do not depend on the choice of the sequence tti > 7r2 > • • • ^ 0. 

Throughout in this paper all relevant quantities such as H_{R), H{R) are allowed 
to take values ±oo. 

In order to establish fundamental formulas for "Large Deviation Principle" (=LDP), 
we need here the following notion. 
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Definition 2.2 (iJ-tight) Let Z = {Zn}^^i be a general source. If 

lim limsup - logPr{|Z„| > K} = -oo (2.6) 

K^oo 7i_»oo n 

holds, then we say that the source Z = {Zn}'^^i is exponentially tight (abbreviated as 
E-tight; cf. Dembo and Zeitouni [4])- 

Here, wc have the following fundamental theorem for LDP, although it has a rather 
conventional form. This theorem can also be regarded as forming a pair with Theorem 
2.2 below. 

Theorem 2.1 If a general source Z = {Z„}^i is E-tight, then for any measurable 
set r it holds that 

- inf H(R) < limsup - logPr{Z„ e F} < - inf H(R), (2.7) 

where T° , T are the interior and, the closure ofT, respectively. 

Remark 2.1 As will be seen in the proof below, the lower bound in inequality (2.7): 

- inf H{R) < limsup - logPr{Z„ G T} (2.8) 

fiGr° — 'n 

holds without the E-tightness assumption. On the other hand, as for another type of 
upper bound for (2.7), see Remark 5.7 in Section 5 ( also see Corollary 2.2 below). 

Remark 2.2 The E-tightness (Definition 2.2) of the source has the following mean- 
ing for the lower IS rate function II_{R): let K > Q be an arbitrarily large number, 
then for any R such that \R\ > K it holds that 

limsup - logPr{|Z„| > K} > -H{R) 

n— *oo ^ 

in view of the definition of H_{R) . Hence, 

limsup-logPr{|Z„| > is:} > - inf H(R). 

n^oo n \R\>K 

Therefore, from the E-tightness (2.6) we have. 

lim inf HIR) = +oo, 

K^oo\R\>K 

liminf fl:(i?) = +CX), liminf ^(ii) = +oo. (2.9) 

R-^+oo R—*-—oo 

HoweverC(2.9) does not necessarily imply the E-tightness. For example, if we con- 
sider the source Z = {Zn}^^i such that Pr{Zn = n} ^ Pr{Zn = — n} = i (Vn = 
1, 2, • • •), this leads to H_{R) = II{R) = +oo (Vi? G R), and so in this case (2.9) holds 
but (2.6) does not holdD □ 



that is. 
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Proof of Theorem 2.1: The proof is quite elementary. It is enough to show 

limsup - logPr{Z„ G T} < - mi_H{R) (2.10) 

and (2.8). 

a) First we show (2.8). For notational simphcity we use here the notation rg{R) 
= {R- SCR + S) (cf. (2.1)). Then, for any small number ^ > it holds that 

Ts{R)dMR) (V«>«o('^)). (2.11) 
On the other hand, from the definition (2.4) of H_{R) it follows that 

limsup - logPr{Z„ e ^i{R)} > -{H{R) + 7) (Vi > io(i?)), (2.12) 

n — ^00 ^ 

where 7 > is an arbitrary small number. Therefore, there exists a sequence ni < 
n2 < • • • ^ 00 (dependent on R) such that 

— logPr{Z„, G $i(E)} > -{H{R) + 27) (Vi > io{R)). (2.13) 

Thus, in view of (2.11) and (2.13), 

— logPr{Z„, € Ts{R)} > -{H{R) + 27). 
nk 



Hence, 



limsup - logPr{Z„ e Ts{R)} 

n— >oo ^ 

> limsup — logPr{Z„, G TsiR)} 

fc— >oo n.k 

> -(fl:(ii) + 27). 
As 7 > is arbitrary, we have 

limsup - logPr{Z„ G TsiR)} > -H{R) (V(5 > 0). (2.14) 

n—^oo n 

Here for any J? G r° let 5 > be small enough to satisfy Ts{R) C r°. Then, by 
means of (2.14), 

1 1 

Hm sup - log Pr{Zn G P} > lim sup - log Pr{Z„ G P" } 

> lim sup - log Pr{Z„ G P5 (-R) } 



> -H{R). 
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As R is an arbitrary internal point of r° , we conclude that 

limsup - logPr{Z„ S T} > - mi^H{R). 

b) Next we will show (2.10). It follows from the definition (2.4) of H{R) that 
^ logPr{Z„ e ^iiR)} < -{H{R) - 27) (Vn > no{R, i); Mi > io{R)), 

(2.15) 

where 7 > is an arbitrarily small number. For any constant ii" > set Tk = 
r n [—K,K] and consider an arbitrary R G Tk, then (2.15) with i = io = io{R) 
reduces to 

^logPr{Z„e$o(i?)}< -(«:(-«)- 27) (Vn>no(i?,io);VJ?Gr^), (2.16) 

where wo have put ^o{R) — ^ig{R). It is evident that ^o{R) is an open set. Now we 
notice that Tk is a bounded closed set and hence satisfies the compactness (owing 
to Hcinc-Borcl theorem). As a consequence, we can choose a finite number of points 
i?i , i?2, • ■ ■ ; Rtuk ill so that 



Therefore, for 



we have 



Vn > max(no(iii,io),no(ii2,io),- • ■ ,no{RmK^io)) ■ 



^logPr{Z„ eTif} 

1 f""" \ 
^ -log($^Pr{^nG$o(i?0}l 

< - log ( max Pr{.Z„ G i>o(i?i)} I + - ^ogrriK- 
n \i<i<mK ] n 



Hence, by (2.16) we have 
1 



iogPr{z„ er^} 

n 

< — min iJ(i?;) H — losrriK + ^'y 

l<l<mK n 

< - iiif ^(ii) + - log mif + 27, 

ReTK ^ 
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from which it follows that 



limsup - logPr{Z„ e r^} < - inf H{R) + 27. 

„_>(x) n ReTK 

Then, as 7 > is arbitrary, 

limsup - logPr{Z„ er^} < - inf H{R). (2.17) 

On the other hand, the assumed -E-tightness condition of Z = implies that 

for any large L > 0, 

limsup-logPr{|Z„| > K} < -L {\/K > Kq{L)). (2.18) 

Hence, combining (2.17) and (2.18) results in 

limsup - logPr{Z^ G T} 

< limsup - log (Pr{Z„ e r^} + Pr{|Z„| > K}) 

< limsup ( - logmax (Pr{Z„ e rK},Pr{|.^„| >K}) + - log2 
= limsup ( ma:x: ( -logPr{Z„ e Tk}, - logPr{|Z^| > K} 
= max (limsup — log Pr{Z7i G r^ljlinisup — logPr{|Z^| > K} 

< max(- inf H{R),-L 

\ R^Tk 



min ( in_f H{R),L 
\RerK 



< - mm{ M_H{R),L] . (2.19) 
We notice here that L > is arbitrarily large, so letting L ^ oo we have 

lim sup - log Pr{Z„ G F} < lim sup - log Pr{Z„ e T} 

< - mi_H{R), 

Rer 

thus completing the proof of (2.10). □ 

Let us now proceed to show the second fundamental theorem on large deviation. 
To do so, we need the following notion. 
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Definition 2.3 (cr-convergence) Let fn{R) {n = lj2,---) be a sequence of functions 
of R on R. If for any bounded closed subset VofTL and for any 7 > there exists a 
sequence m < n2 < • • • — > +00 (independent of R G V) such that 

fn,{R) > limsup/„(ii) - 7 (Vfc > koiR,7y,yR e V), (2.20) 

n — *oo 

then we say that {/ra(-R)}JJLi is a-convergent. 

Remark 2.3 It is easy to check that {/n(J?)}^i is a-convergent if {fn{R)}^=i con- 
verges on R. As for the IS meaning of Definition 2.3, refer to Definition 2.4 below. 

□ 

Definition 2.4 (cr-convergent) Given a general source Z = {Zn}^^i, set 

fn{R) = ^ log prj^ g$ (i?)} ""^ ^^^'^ 

where ^i{R) is defined as in ( 2.1). If {fn{R)}'^=i is a-convergent in the sense of 
Definition 2.3, then we say that the source Z = {Zn}^^i is a-convergent . □ 

Remark 2.4 It is not difficult to check that II{R) = H_{R) (Vi? € R) is a sufficient 
condition for the a-convergence o/Z = {Zn}'^^^. □ 

With these definitions, we have the following second fundamental theorem, which 
can be regarded as providing the pair with Theorem 2.1. 

Theorem 2.2 // a general source Z = is E-tight and a-convergent, then 

for any measurable set T it holds that 

- inf H(R) < liminf - logPr{Z„ € T} < ^ inf H{R), (2.22) 

-RGr° n-too n ii^Y' 

where T° , T are the interior and the closure ofT, respectivelyD □ 

Remark 2.5 As will be seen from the proof below, the lower bound in inequality 
(2.22): 

- inf TliR) < liminf - logPr{Z„ G T} (2.23) 

holds without the assumptions of Theorem 2. 2D □ 
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Proof of Theorem 2.2: The proof is quite elementary. It suffices to show 

liminf - logPr{Z„ G T} < - m{_H{R) (2.24) 

and (2.23). 

a) First we show (2.23). Ahhough the proof of this part basicaUy parallels that of 
part a) of Theorem 2.1 with due modifications, we write it again for the reader's 
convenience. Here too, we use the notation that Ts{R) = {R — SCR + 6) {S > 0). 
Then, we have 

TsiR)D^i{R) {yi>io{S)). (2.25) 
Moreover, by the definition (2.5) of H{R), 

liminf - log Pr{Z„ G $i(i?)} > -(S^(i?) +7) (Vi > io(i?)), (2.26) 

n— ►cx) n 

where 7 > is an arbitrarily small number. Hence, 

-logPr{Z„ G ^iiR)} > -(HiR) + 27) (Vn > no(i?,z); Vi > io(i?)). (2.27) 
n 

ThereforeCby (2.25) and (2.27), 

^ logPr{Z„ G Ts{R)} > -(H{R) + 27) (Vn > no{R, 5)). 

Hence, 

liminf - logPr{Z„ G TsiR)} > -(H{R) + 2j). 

n — *oo Ti 

As 7 > is arbitrary, we have 

liminf - logPr{Z„ G Ts(R)} > -H(R). (2.28) 

n— >cx) n 

Now, for any G r° we can choose a small 5 > so that Ts{R) C r°. Then, by 
(2.28), 

Hm inf - log Pr{Z„ G F} > lim inf - log Pr{Z„ G F° } 

n—*oo ft n—*oo fi 

> lim inf - log Pr{Z„ G F5 (i?) } 

> -H{R). 
Since ii G r° is arbitrary, we conclude that 

liminf - logPr{Z„ G Fj > - inf H(R), 
n-yoo n Rer° 

which implies (2.23). 
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b) Next we show (2.24). With an arbitraily large K > we set Tk = T D [-K, K]. 
By the definition (2.5) of H{R), 

H, (i?) = lim sup - log I ^ tm\ 

n^oo n Pr{Z„ e <I>i(i?j} 

> HiR)-"f {yi>io{R)). (2.29) 

Then, by the assumed cr-convergence of Z = {Zn}^=i, there exists a sequence m < 
722 < • • • — > +00 (independent of R & Tk ) such that 

(Vfc > ko{R,iy, Vi > io{R); VR e Tk). (2.30) 

As a consequence, combining (2.29) and (2.30) yields 

Pr{Z„, e ^i{R)} < exp[-nk(H{R) - 

(yk>ko{R,i); \fi>io{R);yReTK). (2.31) 

Consider the special case of (2.31) with i = zq = io{R), and put ^o{R) = $i(,(-R). 
Then, (2.31) reduces to 

— logPr{Z„, e$o(i?)} 
rik 

< -(HiR) - 37) (Vfc > ko{R); Vi? e Tk). (2.32) 

Wc note here that Tk is a boimdod closed set and hence is compact (owing to Hcine- 
Borel theorem). Therefore, there exists a finite number of Ri, R2, ■ ■ • , RmK ^ 
such that 



1=1 



Thus, 



— iogPr{z„, er^} 



1 f""" \ 

^ —^og\J2P^{Zn,^MRi)}j 

< — log ( max TT{Zn^ G ^o{Ri)}] + — logmK, 

Uk \l<l<mK J rife 



from which together with (2.32) it follows that 
— logPr{Z„, GP^} 



< — min H(Ri) + —logrriK + S'y 

l<l<mK Tlk 

< - inf HiR) + — logTUK + 37. 
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Hence, 



lim inf - log Pr{Z„ e r^} < Hm inf — log Pr{Z„, e r^} 



< - inf H{R) + 3'y. 



As 7 > is arbitrary, we have 



liminf-logPr{Z„ e Tif} < - in_f H{R). (2.33) 

On the other hand, by the assumed S-tightness condition of the source Z = {Zn}^^i, 
for any large L > 0, 

limsup - logPr{|Z„| > K} < -L {yK > Ko{L)). (2.34) 

n — >oo ^ 

ThenCfrom (2.33) and (2.34), 

liminf-logPr{Z„ G P} 

< lim inf - log (Pr{Z„ e Vk) + Pr{|Z„| > K}) 

n^oo n 

< liminf (- logmax (Pr{Z„ G rK},Pr{|Z„| > if}) + - log2 

= liminf (max ( - logPr{Z„ G P/f}, - logPr{|Z„| > K} 

n^oo y yn n 

< max (liminf - logPr{.^„ G Pk}, limsup - logPr{|.Z„| > K} 

< max ( - in_f H{R),-L 

V ReTK 

= -min( inf H{R),L 

\RerK 



< -mini M_H{R),L] . (2.35) 
We notice here that i > is arbitrarily large, so letting i — > oo we have 

Um inf - log Pr{Z„ G P} < hm inf - log Pr{Z„ G P} 

n^oo Tl n^Qo n 

< - mi_H{R), 
Rer 

which implies (2.24)D □ 
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So far we have demonstrated two fundamental formulas for large deviation ( The- 
orem 2.1 and Theorem 2.2), which are quite basic from the viewpoint of information- 
spectra. It should be noted here that 

lim sup - log Pr{Z„ e T} and lim inf - log Pr{Z„ e T} 

n^oo n n^oo n 

arc both given tlwur own lower and upper bounds, respectively, while in usual large 
deviation theorems limsup„^;^ ilogPr{Z„ £ F} and liminf„^oo ^logPr{Z„ e F} 
are altogether given a pair of lower and upper bounds. 

From this conventional standpoint, we can specialize Theorem 2.1, Theorem 2.2 
along with Remark 2.1, Remark 2.5 to the following two corollaries {full LDP and 

weak LDP): 

Corollary 2.1 (Full LDP) Let a source Z = {Zn}^^^ he E-tight, and suppose that 
the IS rate functions have the limit H{R), i.e., H{R) = H{R) = H{R) (iR e K). 
Then, for any measurable set T, 

- inf H(R) < liminf-logPr|Z„ e Fj 

< limsup - logPr{Z„ € F} < - mi_H{R). (2.36) 

Remark 2.6 The E-tightness condition in Corollary 2.1 is actully needed in order 
to make the full LDP hold.. For example, let us consider the source Z = {Z„}^i as 
was shown in Remark 2.2. Then, 

H{R) = H{R) = +00 {WR e R), 
lim - log Pr{Z„ G F} = (F = R), 

n — *oo Tl 

which obviously contradicts (2.36). □ 



Corollary 2.2 (Weak LDP: Dembo and Zeitouni [4]) Suppose that the IS rate func- 
tions have the limit II{R), i.e., II{R) = II_{R) = II{R) (Vi? G R^. Then, the following 
weak LDP holds: 



1 ) For any set F, 



inf HiR) < lim inf - logPr{Z„ e F}; (2.37) 



2) For any compact set T, 



limsup - logPr{Z„ G F} < - inf H{R). (2.38) 
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3 Examples 

In this section, as iUustrative examples of the fundamental formulas as shown in the 
foregoing section, let us consider the large deviation behavior of mia;ec? sources and/or 
nonstationary sources (cf. Han [3]). 

A. Mixed sources: 
First, let 

Xi = (Xi,i,Xi,2,---), (3.1) 

X2 = (X2,1,X2,2,---) (3.2) 

be two stationary mcmoryless Gaussian sources with values in R, and let the proba- 
bility distributions of 

= (Xi_i, Xi_2, • • • , -'^l.n), 
^2 = (-^^2,1, ^2,2, • • • , -^^2,71) 

be denoted by Px'^{-), Px"{-), respectively. Moreover, let the mia;ed source of Xi and 
X2: 

X=(Xi,X2,---) (X" = (Xi,X2,---,X„)) (3.3) 
be defined as the source with the probability distribution: 

Px" (dx) = aiPx^ (dx) + a^Px^ (dx) (n = 1, 2, • • • ; x G R"), (3.4) 

where ai > 0, q;2 > are constants such that ai + a2 — 1- The mixed source X thus 
defined is not memoryless but stationary. 
Setting 



1 

Zn = -yXi, (3.5) 

i=l 

we are interested in the large deviation behavior of Z = {Zn}^=i- Put 

1 " 

Zl,n = -Yl ^1.^' (^-6) 



n 

i=l 



,n = ^i2^2,i (3.7) 



1=1 

and with a fixed number Rq let $i(i?o) be such as defined in (2.1) in Section 2D First, 
by Cramer's theorem for the arithmetic mean of a stationary memoryless source (cf. 
Dembo and Zeitouni [4]), we have 

- inf Ii{R) < liminf-logPr{Zi„ G $i(Po)} 

fl6*i(flo) "^00 n 



< limsup-logPr{Zi,„ G $i(i?o)} < - inf Ii{R), 

n—yoo n fle*i(JJo) 



(3.8) 
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where Ji (R) is the large deviation rate function of the stationary memoryless source 
Zi = {Zi^n}?^=i that is defined as 

Ii{R) = sup{eR - ipi{e)) (3.9) 

9 

in terms of the cumulant generating function ipi{9) = logE(e^"^i'i) of Xi i. Since 
Ii{R) is a closed convex function ( and hence a continuous function) of R and $j(i?o) 
is a nonempty open interval, we have 

inf Ii{R) = inf h{R). 

Re^iiRo) Re^iiRo) 

Then, by virtue of (3.8), we see that ^ logPr{Zi_„ e $i(i?o)} has the limit (as n — > 
oo), and (3.8) is written as 

lim -logPr{Zi,„e$,(i?o)} = - inf Ii{R). (3.10) 
n^oo n iJe*i(flo) 

In an analogous manner, we have 

lim ilogPr{Z2,„e$,(i?o)} = - inf hiR), (3.11) 

n-><X) n Reii{Ro) 

where I2 (R) is the large deviation rate function of the stationary memoryless source 
Z2 = {Z2,n}?^=i that is defined as 

hiR) =sup{e R-^O)) (3.12) 
e 

in terms of the cumulant generating function (p2{0) = logE(e^'^^'^) of ^2,1. On the 
other hand, in view of (3.4) we see that 

Pr{Z„ e $i(iio)} = ai Pr{^i,n G $i(i?o)} + ^2 Pr{^2,n € $i(iio)}, 

from which together with (3.10), (3.11) it follows that i logPr{Z„ e <^^{Ro)} has the 
limit such as 

lim -logPr{Z„ e $,(i?o)} = -minf inf h{R), inf hiR) 
n^oo n XRe-^iiRo) Re^i{Ro) 

(3.13) 

As a consequence, from the definition of H_[R), H{R) as in Section 2, we have 
H{Rq) = - lim lim -logPr{Z„ e $i(i?o)} 



= lim min ( inf Ii(R), inf I2(R) 

i^<x \Re^i{Ro) fle*i(iJo) 

= mm{h{R^),l2{Ro)), 
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where we have again invoked the closed convexity (and hence the continuity) of the 
functions Ii{R), hiR) (cf. Rockafeller [6] ). Therefore, 

H{R) = min(/i(i?),/2(i?)) (Vi? G R). (3.14) 

Similarly, 

H{R) = min(7i(ii), hiR)) {VR G R). (3.15) 

These (3.14), (3.15) are the lower/upper IS rate functions of the arithmetic mean 

Z = {Zn}'^^i for the mixed source X. It should be remarked here that /(i?), hiR) 
are convex functions but min(/i(i?), J2(-R)) is not necessarily convex, which means 
that the rate functions H_{R), H{R) are not necessarily convex. Also, it is easy to 
check that the source Z = {Zn\'^^i is i^-tight. Thus, Corollary 2.1 together with 
(3.14), (3.15) yields the large deviation formula in the case of mixed sources: 

- inimm{h{R),hm 

< lim inf - log Pr{Z„ e T} 

n— >oo fi 

< lim sup - logPr{Z,i G T} 

n— *oo ^ 

< - M_mm{Ii{R),l2{R)). (3.16) 

ReT 

On the other hand, we recall that the cumulant generating function of Z = 
is given by 



(/^„(^) = ^log J Pz„{dz)e"^^ 



which is here written as 



^n{9) = ^ log/ {(^iPz,Jdz)e^^' + a2Pz,,Adz)e"'') . 
It is not difficult to verify that the limit (p{9) = lim„^oo <^n(^) exists with 

ip{e)=im^{M9),M9))- (3.17) 

We notice that the functions ^{0), ipi{9), (p2{9) are always convex. With this ip{6) let 
us here define, as usual, the "rate function" by 

I{R) = snp{0R-ip{e)). 

9 

However, this "rate function" I{R) is always convex and hence is different from the 
lower/upper IS rate functions II{R),H{R), because H_{R),H{R) are not necessar- 
ily convex. Thus, in the case of mixed sources I{R) cannot be a "pertinent" large 
deviation rate measure. 

B. Nonstationary sources: 
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Here too, we consider two Gaussian sources 

Xi = (Xi,i,Xi,2,---), (3-18) 
X2 = (X2,1,X2,2,---) (3.19) 
such as defined in (3.1) and (3.2). Define the source Z = {.^n}^i by 

{Zi,n = ^ E"=i if is odd, 

(3.20) 
Z2,n = ^ T.7=i ^2,i if n is even. 

Then, it is not difficult to verify that 

H(Ro) = Hm Um sup — loe: — — r— , „ ^ 

= max(i?i(i?o),i?2(i?o)), (3.21) 

where H{Ro), H i{Ro) , H 2{Rq) are the upper IS rate functions for Z = {Zn}^=i, 
Zi = {^i,n}^=i, Z2 = {^2,n}^=i, respectively. 
Similarly, it is not difficult to verify also that 

1 1 
H_{Ro) = lim lim inf — log 



= mmiH,iRo),HM)), (3-22) 

where H_{Ro), H_i{Rq) , H_2{Ro) are the lower IS rate functions for Z = 

Zi = {2'i^„}^^i, Z2 = {^■a,™}^^!, respectively Thus, we see that H{Ro) 7^ Hi{Ro) 

in general. 

Now, (3.21) and (3.22) are rewritten as 

H{Ro) = min(/i(7?o),/2(i?o)), (3.23) 

H{Ro) = max{h{Ro),l2{Ro)), (3.24) 

because 

H^{Ro) = Hi{Ro) = hiRo), 
H^iRo) = H2{Ro) = h{Ro). 

Since it is easy to check that the source Z = is i^-tight, C-tight and a- 

convergent. Theorem 2.1 and Theorem 2.2 yield 

- inf min(7i(i?),/2(ii)) 

< lim sup - logPr{Zn G F} 

n— *oo ^ 

< - intmin(/i(E),/2(i?)), (3.25) 

Rev 
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and 



- inf max/i(i?),/2(i?)) 

< lim inf - log Pr{Z„ e T) 

n—^oo fl 

< - mi_ma^{h{R),l2{R))- (3.26) 

Rer 

Thus, in this nonstationary case, the large deviation principle cannot be specified only 
with a single rate function but can be with a pair of rate functions as in (3.25 ) and 
(3.26). Notice that Corollary 2.1 for this case does not work as well. 



4 Note on Generalizations 

So far wc have established two fundamental theorems (Theorem 2.1, Theorem 2.2) 
assuming that random variables Z„ take values in the real space R. Actually, however, 
we can generalize these theorems to the case where Z„ takes values in a general 
topological space X. To see this, we extend Definitions 2.1, 2.2, 2.4 as follows: 

Definition 4.1 * 

H{R) = sup lim sup - log \ , (4.1) 

H{R) = sup lim inf - log ^ I , (4.2) 

where sup^(^) denotes the supremum over all the neighborhoods v{R) of R. □ 



Definition 4.2 (iJ-tight) If for any L there exists a compact set Al C X such that 

lim sup - log Pr{Z„ & Al} < -L (4.3) 

n— >oo n 

where c indicates the complement of a set, then we say that the source Z = {Zn}^=i 
is exponentially tight (abbreviated as E-tight; cf. Dembo and Zeitouni [4])- □ 



Definition 4.3 (a-convergent) Given a general source Z = {Zn}^^i, set 

fn{R) = — log — -, ^ — — r-r [with any fixed neighborhood v{R)). (4.4) 

n P'riZn e v[R)} 

If {fn{,R)}'^=i is cr-convergent in the sense of Definition 2.3, then we say that the 
source Z = {Zn}^^i is a-convergent . □ 



*This definition is found in Dembo and Zeitouni [4] 
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Remark 4.1 As will be easily seen, Definitions -^.i, Definition 4-2, Definition 4-3 
reduce to Definition 2.1, Definition 2.2, Definition 2.4, respectively, in the case of 
X = Ti. □ 



Theorem 4.1 With these extended definitions, Therem 2.1 and Theorem 2.2 as well 
as Corollary 2.1 and Corollary 2.2 hold also with any Hausdorff topological space X 
instead o/R. 

Proof. It suffices basically to parallel the proofs of Theorem 2.1 and Theorem 2.2 
with clue iiiodific-ations, while paying attention to the fact that the intersection of a 
closed set S and a compact set T is compact in the Hausdorff topological space X. □ 



5 Cumulant Generating Functions and Information- 
Spectrum Rate Functions 

Thus far having established the fundamental formulas ( Theorem 2.1, Theorem 2.2, 
Corollary 2.1, Corollary 2.2) on general large deviation problems described in terms of 
the lower/upper IS rate functions H_{R), H(R), we are now interested in the problem 
of how to compute H_{R), H{R) when the random variables Zn take values in R. 

In many "simple" source cases, for example, as Cramer's theorem and Gartner- 
Ellis' theorem tell us, a desirable large deviation "rate function" I{R) is computed 
as the Fenchel-Legendre transforms (cf. Rockafeller [6]) of the cumulant generatiing 
function ip{6) (or something like that): 

7(i?) = sup(6'i?-(^(6l)). (5.1) 



In such cases, the problem of computing the large deviation function I{R) reduces 
to how to compute the cumulant generating function ^p{9). On the other hand, we 
notice here that, in the light of Theorem 2.1 and Theorem 2.2, H_{R) and H{R) also 
should be regarded as "rate functions," which suggests that H_{R), H{R) might be 
set to be equal to I{R) as: 

H{R) = H{R) = sup{eR - cp{e)). (5.2) 



However, in most general cases that are not necessarily elementary and/or typical, 
the right-hand side of (5.1) does not give rise to a desirable large deviation rate 
function any more, as was already seen in the foregoing section, i.e., in general cases, 

H{R),H{R) sup{eR - <fi{e)). (5.3) 

9 

Our main concern with (5.2), (5.3) then addresses the problem of how to elucidate 
under what conditions (5.2) holds and/or under what conditions (5.2) does not hold; 
furthermore, also if not then how not. 
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In this section we address this problem. Let Z = {Zn}^=i be a general source. 
To this end, let us start with any fixed closed interval M = [Mi, M2] (Mi < M2) and 
define the M-truncated cumulant generating function by 



^lf\e)^-log I PzAdz)e^'', (5.4) 
'm 



and also, define 



^M(^)=limsup^(f)(^^) (V^eR), (5.5) 

n^oo 

^^(^)=liminf<p(f)(e) (V^eR). (5.6) 
Moreover, define the M-truncated lower/upper IS rate functions H_j^{R), Hm{R) by 



H{R) 


for 


ReM, 


+00 


for 


R^M, 


H{R) 


for 


ReM, 


+00 


for 


R^M, 



Hm{R) = < V ' 7 D^ »7 (5-8) 
^ \ +00 for i? ^ M, ^ ^ 

where H_{R), H{R) are the lower/upper IS rate functions. Then, we have the following 
fundamental formulas. 

Theorem 5.1 For any general source Z = {Z„}^i it holds that 

JpM{e) = supi9R-HM{R)) (V^eR). (5.9) 

R 

<fi^{e) > snp{eR-HM{R)) (V^eR). (5.10) 

R 

The proof of Theorem 5.1 is given in Section 6. □ 



Remark 5.1 In case Z = {Zn}^-i is a-convergent (cf. Definition 2.4) in Theorem 
5. 1, the following holds: 

^j^{e)^sMOR-HM{R)) (V^eR). (5.11) 

R 

Let us now consider the special case of (5.4)^ (5-11) with M = [—K,K] (K > 0). 
This special case is indicated, with an abuse of notation, simply by "iC" in place of 
"M." Then, an immediate consequence of Theorem 5.1 with K in place of M (under 
the limiting operation K — > 00) is the following theorem, where we have set 



= lim ^^,{6) {Ve G R), (5.12) 

K— >CX) 

<^°(^) = lim <fi {6) {Ve e R). (5.13) 

— K^oo — 
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Theorem 5.2 For any general source Z = {^n}^i it holds that 

lp°{0) = sup{eR-H{R)) (V6'eR), (5.14) 

R 

<fi°{e) > iiup{eR-H{R)) (V^IgR). (5.15) 

Remark 5.2 In case the a-convergence property is satisfied in Theorem 5.2, the fol- 
lowing holds: 

ip°{e) = sup{eR - H{R)) {ye e r). (5.16) 

~ R 

Proof of Theorem 5.2: 

In view of Theorem 5.1 it suffices to take account of the definition of <f°{9), (p°{6) 
and to notice that 

lim sup{eR - Hj^{R)) = sup{eR-H{R)) (V6' e R), (5.17) 

lim sup{eR-HKR)) = sup{0R-H{R)) {W eR). (5.18) 

□ 

So far in Theorem 5.1 we have shown a relation between the M-truncated cumulant 
generating functions and the M-truncated lower/upper IS rate functions. We now 
want to see the direct {not via M-truncation) relation between the non-truncated 
cumulant generating functions and the non-truncated lower/upper IS rate functions. 

The non-truncated cumulant generating functions are defined by 

(^„(0) = -log/ Pz„(^^^)e"'^ (5.19) 
n J-oo 

and 

lp{e) =\imsup(pn{0) (V6IGR), (5.20) 

n— J'OO 

ip{6) = liminf (/j„(6l) (V6' G R). (5.21) 

Paralleling the previous functions ip^\6), 'p°{0), we define the following "tail" func- 
tions with an arbitrary K > 0: 

^(vi^)(0) ^ iiog/ Pz„(d^)e"'^^ (5.22) 
n J\z\>K 



lp'{e) = lim lim sup ¥'^^^^(61). 

Definition 5.1 ( C-tight) Let 7i = be a general source. If 

lim ifi^'^'^\0)=-oo {ye&R), (5.23) 

K—^00 
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then we say that Z = {Zn}^=i is cumulatively tight (abbreviated as C-tight). □ 

Then, wc now have the following lemma that relates the truncated cumulant gen- 
erating functions to the non-truncated cumulant generating functions: 

Lemma 5.1 // a general source Z = {Zn}^=i is C-tight (cf. Definition 2.2), then it 
holds that 

r{0)=m, '£°{6)=^{e) (V^eR). (5.24) 

The proof of this lemma is given in Section 6. □ 



Now, Lemma 5.1, together with Theorem 5.2 and Remark 5.2, immediately leads 
to the following Theorem 5.3 and Remark 5.3, respectively. 

Theorem 5.3 If a general source Z = {Zn}^^i is C-tight, then 

<p{e) = sup{eR-H{R)) (V(9gR), (5.25) 
(p{0) > sup{0R-H{R)) (V6'gR). (5.26) 

~ R 

Remark 5.3 In case Z = {Zn}^^i is C-tight and a-convergent in Theorem 5.3, the 
following holds: 

ipiO) = sup{eR - HiR)) {\/e G R). (5.27) 

~ R 

Remark 5.4 Theorem 5.3 along with Remark 5.3 is reminiscent of Varadhan's inte- 
gral lemma [7] in which the linear function 6R of R in the former is replaced by an 
arbitrary continuous function 0(ii) on a regular topological space X . However, the 
former does not follow as a special case from the latter, because in the latter case the 
existence of a good rate function is assumed. □ 



Before proceeding to show the IS formulas for rate functions described in terms 
of the cumulant generating functions, we need two definitions and one lemma. 

Definition 5.2 (Rockafeller [6]) Given a function f on R, we define the closed convex 
hull function Uf of f as the pointwise supremum of the collection of all affine functions 
h onK such that h{R) < f{R) (iReK). It is evident that Uf{R) < f{R) for all 
i? G R. IfUf{R) = f{R) for all RgR, we say that f is a closed convex function. 

Definition 5.3 (Fenchel-Legendre transform: Rockafeller [6]) // 

g{e) = sup{eR-f{R)) (V^gR), (5.28) 

R 

then we say that g is the conjugate of f , and denote the g by f*. 

Lemma 5.2 (Rockafeller [6]) The conjugate f* is always a closed convex function. 
Moreover, it holds that f* = (U/)* and /** = Uf. □ 
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Thus, applying Lemma 5.2 to Theorem 5.3 and Remark 5.3 immediately yields 
the following inverse formulas: 

Theorem 5.4 (Inverse formula) For any C-tight source Z = {Zn}^^i it holds that 
\JH{R) = sup{eR-ip{e)) (ViiGR), (5.29) 

9 

UH{R) > snp{0R- ip{e)) (Vi?eR). (5.30) 

9 ~ 

Remark 5.5 (Inverse formula) In case the a-convergence property is also satisfied in 

Theorem 5.4, the following holds: 

UH{R) = sup(6'i? - ip{e)) (VR e R). (5.31) 

9 ~ 

Remark 5.6 Theorem 5.4 as well as Remark 5.5 is reminiscent of Bryc's inverse 
Varadhan lemma [8] in which the linear function OR of R in the former is replaced 
by an arbitray bounded continuous function f{R) on a completely regular topological 
space X. However, the former does not follow as a special case from the latter, because 
linear functions are not bounded. 

Furthermore, Theorem, 5.4 as well as Remark 5.5, together with Theorem 5.3 along 
with Rem,ark 5.3, is reminiscent of Dem,bo and Zeitouni [4, Theorem 4-5.10], in which, 
however, the existence of a good rate function is assumed unlike in our case. □ 

A direct consequence of Theorem 5.3, Theorem 5.4, Remark 5.3 and Remark 5.5 
is the following corollary which states a key relation between the lower/upper IS rate 
functions H_{R),H{R) and the cumulant generating functions if{0),Tp{0): 

Corollary 5.1 (A condition for the normalized cumulant generating function to have 
the limit) 

1) Let a source Z = {Zn}^^i be C-tight. Then, 

\JH{R) = UH{R) (Vi? G R) (5.32) 

implies 

ip{0)=.ip{0) (V6'eR), (5.33) 
that is, the cumulant generating function ipn{0) defined by (5.19) has the limit 

if{0) = 'f{0) = Tp{0) (V0 e R). (5.34) 

2) If a source Z = {Zn}^^i is not only C-tight but also is a-convergent, then 
(5.32) is the necessary and sufficient condition for (5.33). □ 

Proof 1) Suppose that (5.32) holds. By Lemma 5.2 combined with Theorem 5.3 we 
see that 

^(6*) = UipiO) ^ sup{OR - UH{R)) for V6I G R, (5.35) 

R 

ip{0) > Uip{0) > sup{OR - UH{R)) for \/0 G R, (5.36) 

~ ~ R 
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which together with (5.32) yields = (V0 G R), where we have used the fact 
2) Suppose that (5.34) holds. Then, from (5.29) and (5.31) we have (5.32). □ 

In some sense, formulas (5.29) and (5.31) may be regarded as providing formulas 
for computing the lower/upper IS rate functions H_{R), H{R) as the Fenchel-Legendre 

transforms of the cumulative generating functions (p{9),Tp{0). To see this more, let us 
define the following rate functions X(-R), I{R) of Cramer-Gartner-Ellis type by 

1{R) = sup{eR - Tp{e)) {VR e R), (5.37) 
e 

1{R) = aupiOR - ip{e)) (Vi? G R). (5.38) 

9 ~ 

Then, from (5.29) and (5.31) wc have 

UH{R) = I{R), UH{R) = 7{R) (Vi? G R). (5.39) 

However, in view of Theorem 2.1, Theorem 2.2 and Corollary 2.1, Corollary 2.2, the 
formulas that we wanted to obtain were those for computing IJ_{R) =H{R) but not 
for UH{R),UH{R). Formula (5.39) tells us that the "rate function" I{R),I{R) can 
capture, as well, relevant structures of large deviation probabilities that arc reflected 
via the nature that £(R), I{R) are closed convex functions (cf. Lemma 5.2). In other 
words, I_{R),I{R) overlook all the finer structures that cannot be grasped via the 
closed convexity of /(i?), /(i?) alone. We should be reminded that H_{R),H{R) are 
not necessarily closed convex functions. 

Thus, we do not yet reach the relevant formulas for computing II{R), H{R) via 
the cumulant generating function, which remains to be further investigated. On 
the other hand, even without such relevant computation formulas, we could enjoy 
insightful general view, demonstrated so far in this paper, at basic large deviation 
problems. This is an advantage of the IS approach. 

These observations can formally be summarized as: 

Theorem 5.5 (Reduction theorem) Let a general source Z = {Zn}'^-i be C-tight 

and a-convergent. Then, it holds that 

UH{R) = I{R), UH{R) = 1{R) {WR G R). (5.40) 
In other words, it holds that 

1) H{R) = I{R) (Vi?eR) (5.41) 
if and only if H_{R) is closed and convex; and also that 

2) H{R) = I{R) (Vi?GR) (5.42) 

if and only if H{R) is closed and convex (cf. Definition 5.2). ( Thus, in this case, the 
computation problems for H_{R), H{R) completely reduces to those for L{R), 
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Proof: The former part is the same one as in (5.39). The latter part follows if we 
observe that L{R), I{R) are always closed convex functions. □ 



Theorem 5.6 (General note) Let Z = {Zn}^^i be a general source and ip{6) be the 
cumulant generating function defined by (5.19), (5.20). Then, 

H{R) > UH{R) > I{R) {yR e R), (5.43) 

where I_{R) was defined in (5.37). 

Proof. The Fenchel-Legendre transformation of (5.14) gives 

UH{R) = svi-p{0R-<^{9)), 
e 

from which together with Tp[6) > ^°{0) it follows that 

H{R) > UH{R) > sup{eR - Tp{e)) = I{R). 

6 

Thus, in this general case, the computation problem for H_{R), H{R) does not reduce 
to that for 1{R). 

Remark 5.7 It is evident thatI_{R) is a closed convex function. Application of (5.4.3) 
to the right-most term in Theorem 2.1 yields a Cramer-Gartner-Ellis type of upper 
bound (though in general much looser): 

limsup - logPr{Z„ G T} < - inf UR) (5.44) 
n^oo n Rer 

for any compact set T (cf. Dembo and Zeitouni [4]). □ 



The following final remark concerns the "locality" of the truncated Fenchel-Legendre 

transforms: 

Remark 5.8 (Locality) Let a source Z = {Zn}^^i be C -tight. We define Lm{^) 

and suppose that " closed convex function, i.e., UH_{R) — H_{R). Then, from 

(5.29) we have H{R) = I{R), and hence H^iR) = Lm{R) {yM,\/R e R). On the 
other hand, the he Fenchel-Legendre tansform of (5.9) turns out to be 

U{Hj^){R)=snp{eR-ipM{R)). 
Thus, in view of \J{H_j^){R) = I^j^{R) = £j^{R), we obtain 
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Im{R) = supiOR - ^mW) (VM = [Mi,M2],yR G R). (5.46) 

9 

Then, (5.46) means that, if we want to calculate the value of the rate function /(i?) 
at some R = ao, it is not necessary to calculate the values ofTp{6) over all 6 € H 
and transform it. Instead, choose a small interval (c, d) contaning ao then it suffices 
to compute the cumulant generating function ^m(^) "^^^^ over the domain M = [c, d\ 
no matter how small it is. This demonstrates the "locality" of the rate function I_R). 
Similarly for I{R), ip{9). □ 



6 Proofs 

In this section we give the proofs of Theorem 5.1 and Lemma 5.1. 
6.1 Proof of Theorem 5.1 

The proof of Theorem 5.1 consists of several steps, though they are elementary. 
The mainstream is to directly compute the cumulant generating function with M = 

[Mi,M2] (Ml <M2): 

^(M) (^) = 1 log /■ {dz)e^'^ (6.1) 

in terms of the quantities 7ri,7r2, • • S ^i{R)j H i{R) , H_^[R) , H[R),H_{R) defined as in 
the beginning of Section 2. 

Step 1: 

For each i = 1, 2, ■ • •, set tti = 2~' and define an open interval ^i{R) ( cf. Section 
2) by 

$,(i?) = (i?-7r„i? + 7r,). 

Then, since M = [Mi,M2] is compact, for each i there exists a finite number Lj of 
open intervals 

$i(ap'^) (j = l,2,...,ii) (6.2) 

such that 

a'f^GM (j = l,2,---,Li) (6.3) 

and 

Li 

Mc\J M4'^)- (6-4) 

The collection (6.2) of such open intervals is called a finite cover of M, simply denoted 
by 
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Hereafter, for notatfonal simplicity, we write I^''^ instead of ^i{af^). Then, the 
integral of (6.1) is upper bounded as 

-log [ Pz„(d^)e"^^ < -logV / Pz„(d-^)e"^^ (6.5) 



H, {aY> ) = lim inf - log ^ , (6.6) 



On the other hand, by definition, 

= lim inf — log — 

n^oo n Pr{Z„ e II^'}' 

so that, for an arbitrarily small number d > 0, 

< exp[-n(ff,(ap')) - S)] (Vn > 3np^ Vj = 1, 2, • • • , Li). (6.7) 
ThereforeC(6.5) is evaluated as follows. 

Li 

< ^exp[-n{H^{a\^^) - 6)] explnOa'f^ + m^^\e\] 
j=i 

Li 

= ^exp[-n(S,(ap'^) - 6)] exp[n0a\'^ + n2-'\9\] 



Li 

< Li exp[nmax(6'ap^ - Hi{a^p))] exp[n((5 + 2-'|6i|)]. 

3 



(6.8) 



Substituting (6.8) into the right-hand side (6.5) yields 



)(^) < max(eap) - ^,(af )) + 5 + 2-'\e\ + - logL^. (6.9) 
j n 

We now define the function H_{^^ (R) on R by 

i£S"'(«)={*™ t uVm: ("») 

Then, (6.9) can be written as 

^i^^ {0} < snp{0R - (it!)) +5 + 2-y\ + +- log Li, (6.11) 
R n 
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where sup^ means the suprenium over R. Hence, 

limsup¥'(/^)(6») <sup{0R-h!i^\r)) + S + 2-'\0\. (6.12) 

n— >oo R 

It should be noted here that the function Hj^^^^R) is monotone increasing in i, that 
isC 

Si"^ (i?) < ^Ifi' (i?) (Vi? e R; Vi = 1, 2, • • •). (6.13) 
Therefore, we have the hmit function (as was already defined by (5.7)): 

Hj^{R) = lim H^^\R) (Vi? e R), (6.14) 

where the value +cxd is also allowedD NowCby (6.12), 

lim sup (^(f) (61) < Mm. (snv{eR-H^^\R) + 5 + 2-'\e\\ 

< lim sup(^i? - h!^'^^ {R}) + 6. (6.15) 

i — *oo jj- 

Since 5 > is arbitrarily smallCit follows from (6.15) that 

TPm{0) = lim sup 93 W {9) < lim sup(6iii - H^^'^ {R)) 

= lim sup(6'i?- Uijf^'(i?)), (6.16) 

where, for a function f{R) on R, the function U/ is defined in Definition 5.2 (also see 
Lemma 5.2). 

Step 2: 
Define 

5« (i?) = eR- UH^^^ (i?) (6.17) 

and set 

=snT>gf{R), (6.18) 

R 

ge= \vmsn^gf{R). (6.19) 

i — *oo 

Suppose here that gg = —oo, then it trivially holds that 

lim sup (eR - H^^^ (R)) = lim sup (OR - UH^^^ (R)) 

< sup {eR-HM{R)). (6.20) 

R 

Next, let us consider the case of gg > — oo and define the set: 

ge{i) = {R€R\gi'\R)>ge}. 
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Then, we see that sup^ on the right-hand side of (6.18) is attained at some point 
R = Rq G M; and g^g\R) is a closed concave function; so that Geii) (i = 1, 2, ■ • •) is a 
sequence of monotone shrinking closed intervals. Therefore, there must exist at least 
a point i?i G R such that 

oo 

Ri e fl^eW 

i=l 

and 

OR^ - UHm{Ri) = lim (ORr - UHI^\Ri)) > ge, (6.21) 
where we have used the monotonicity of UH_^^\r) in i: 

lim U^f ^' (i?) = UHm{R) (Vi? e R) 
Thus, from (6.17)~(6.19) and (6.21), we have 

lim sup (eR - UH!i^\r)] < sup (OR - UHj^{R)) . (6.22) 

Then, taking account of 

sup (eR - uh^^\r)) = sup (or - h^^\r)) , 

sup {OR - UHM{Rj) = sup {OR - Hj^{R)) , 

R R 

we see that (6.22) is equivalent to 

hm sup (or - H^^^^R)) < sup {OR - Hf^{R)) ■ (6.23) 

ConsequentlyCby means of (6.16) and (6.23), it is concluded that 

Tpyi{0) < sup {OR - Hj^{R)) . (6.24) 

R 

Step 3: 

Next, it follows from the definition of H_^{R) as in (2.2) that, for any small 6 > 0, 
there exists a sequence of positive integers n^^ < 112^ <•••—> 00, which may depend 
on 5 > and i, such that 

k 

(Vfc > ko{i, S); > to{d); Vj = 1, 2, • • ■ , L,). (6.25) 
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Then, 



> max (^-^ log {eM-ni'\Ma'f^) + 6)] eMn^^^Oaf^ - j 

= ma^ (^-L log (exp[-4^)(^,(a«) + 6)] exp[n«ea« - n«2-|0|]) 
= max ((^a« - ^,(ap^)) - + 2-|^|)) 



(6.26) 



As a consequence, 



3 

Set 



max^^ > max(ea,^^^ - fl:i(a,^'0) 2"i^|- (6.27) 



VW(^) = ma^^(^,'(^) (nfc = 3i = 1, 2, • • • , i,), 
J "lb 

then (6.27) yields 

> ^^(ec^) H^{a['^)) - 5 - 2-y\. (6.28) 

We observe here that the left-hand side of (6.28) does not depend on the choice of a 
finite cover $i of M, so that 

)(0) > supmax(0af - ^,(af )) - S - 2-y\, (6.29) 

where sup$. means the supremum over all the finite covers $i of M. It is easy to 
check that 

supmax(0ap) - ^^(ap'^)) = sup(0i? - h}^\R)). 

Thus, 

^(f )(0) > sup(0it! - H^t'\R)) -5-2-^\e\. (6.30) 

R 

Noting that "n^f^ > k (Vj = 1, 2, ■ • • , Li) and taking limsup^j^g^ in both sides of (6.30), 
we have 



^MW=linisup¥'(f)(0) 

n— s-oo 
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> linisup<^^(0) 

k^oo 



AM), 



> sup{eR-ir.'"'{R))-s-2-'\e\ 

R 

> supi9R- Hm{R)) -S -2-^\e\, (6.31) 

R. 

where we have taken account of the monotonicity of H_^^\r). Moreover, taking 
account of linii^tx) in both sides of (6.31) and recalling that 5 > is arbitrary, we 
have 

ifMie) > sup{9R - Hm{R)). (6.32) 

R 

Then, it follows from (6.24) and (6.32) that 

Tpm{9) = limsupv?l*^)(6') = sup(6li? - Hm{R)) (V6l G R), (6.33) 

n— »oo R 

which is nothing but (5.9). 
Step 4- 

Accordingly to (6.6), we set 

Hi (af ^ ) = Urn sup - log — ^ (6.34) 

^ ' ' ™ n ^Pr{Z„ G Jp)} 

and define the function if (R) on R as 

Taking account of the monotonicity in i of h[^\r), we obtain the limit function 

Hm{R) = lim Hf\R) (Vi? G R). (6.36) 

Then, in the same manner as in deriving (6.32), we have 

^^(0) = liminf (^(f )(0) > sMOR-HMiR)), (6.37) 

— ™ n— >oo 

which is nothing but (5.10). 
Step 5: 

On the other hand, the opposite inequality: 

<fi^{e) = liminf ^W(^) < sup((?i? - HMiR)) (6.38) 

does not necessarily hold. In the sequel we will show that the assumed cr-convergence 
on the interval D = M (cf. Definitions 2.3, 2.4) is a sufficient condition for (6.38) to 
hold. 
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In view of (6.34) we see that the a-convergence ( with any small 6) means the 
existence of a sequence m < n2 <•••—> oo, which may depend on i and 5 but must 
not on j, such that 

— log ^- ^ > H^{a^/'') - S 

{yk>ko{i,j,S); V»>»o(<5); Vj = l,2,...,Li)- (6-39) 

We rewrite this as 

Pr{Z„, e 4^'^} < eM-nkiHiia'f^) - S)] 

{yk>ko{i,j,6y, yi>io{6y, Vi = l,2,.--,L,). (6.40) 



Then, 



< exp[-nfe(S^i(af ^) - 5)] exp[nfe^aP + nk2-y\] 
= expK(eap' - Hiia'f^))] exp[nfe(5 + 2-'\e\)]. 



Consequently, 



M 

Li 

< 

J = l 
Li 

< cxpK(^ap') - Hiia^^))] expK(^ + 2-y\)] 

< Li expK max(^ap^ - Hi{4^^))] exp[nfc(<5 + 2-y\)] 

3 



ThereforeC 

1 r 



'I'k J M 



< max(eap'^ - Mo!f^) +5 + 2-'\e\ + 

3 

< sup(0i? - i?f '^i?)) + 6 + 2-'\9\ + i^^. 

R nk 



(6.41) 



(6.42) 



(6.43) 
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Hence, 

^19) ^ liminf^W(^?) 



K— >00 

= sup{9R-H\^\R)) + 5 + 2-y\. (6.44) 

R 

Now, taking linii^oo and letting 5 — > in both of (6.44), we have 

^^{0) = hminf (^(f )(0) < hm snp{0R -h[^'\r)). 

Then, in an analogous manner as in the argument for a sequence of monotone shrink- 
ing intervals (Step 2), we conclude that 

ip^{0)<sup{0R-HM{R)) (V^eR), (6.45) 



which together with (6.37) yields 



ip,J0) = liminf (p(^)((9) = sup(^i? - Hm{R)) (V6I G R). (6.46) 

— ™ n—>oo 



6.2 Proof of Lemma 5.1 

Since 



<^„(^) = ilog / PzAdz)e^'' 
n J 

= -log I I PzAdz)e'''^' + [ PzAdz)e"^ 

1^ \J\^\<K J\z\>K 

< - log max ( / Pz„ (d^)e"^^ / Pz„ (d^)e"^^ | + - log 2 
n \J\z\<K J\z\>K J n 

( - log / Pz„ (d2)e"^^ - log / Pz^ (d^)e"^^ ) + - log 2 
J\z\<K n J\z\>K ) n 

max(^W(0),(p(^^)(^))+llog2, 



H0Z 



= max 



we have 

lp{0)@ = limsupiy9„(6') 



= limsup — log / Pz^{dz)e^ 

n — ^oo 1^ J 

< max ( lim sup <p(f ^ {0) , lim sup ^i^'^'> {0) ) . (6.47) 

\ n^oo n— '■oo / 
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Letting ^ oo in both sides of (6.47), it follows from the assumed C-tightness that 

^{0) < max I lim limsup<^£^^(^), lim limsupy^^''^^(^) I 
= max (^° (^) , — oo) 

= Tp°{9). (6.48) 
On the other hand, (p{0) > <^°{0) always holds, so that we conclude 

^(6*)=^° (6*) (ye&Yl). (6.49) 
Furthermore, in the same way as above, we have 
ip(6) = limmi ipn(0) 

— n— *oo 

< max ( liminf (^(^^)(6'),limsup(^(,^'^)(6') ) . (6.50) 
Again, letting — » oo in (6.50) yields 

ip {6) < max ( lim lim inf <^(f ^ (61) , lim lim sup ip^^^^ (6) 

— V K—>QO n— »cxD K—yoo n— vr^ 



^X— *oo n— *oo K—*oo ri—^oo 

= max (93°(0), — oo) 

= v°{e)7 (6.51) 

where we have used again the assumed C-tightness. Since (p{9) > (p°{0) always holds, 
we conclude that 

ip{0)=ip°{9) (V6leR). (6.52) 
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